[Data] Refactor some file-related unit tests #48228
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
bveeramani
requested review from
scottjlee,
raulchen,
stephanie-wang and
omatthew98
as code owners
bveeramani
commented
Oct 23, 2024
Comment on lines
-776
to
-778
| error_message = "No input files found to read" | ||
| with pytest.raises(ValueError, match=error_message): | ||
| ray.data.read_csv(path3, file_extensions=["csv"]).schema() |
There was a problem hiding this comment.
We shouldn't test that this raises a value error, because returning an empty dataset is also valid
scottjlee
approved these changes
Oct 23, 2024
| @@ -846,7 +813,7 @@ def test_csv_read_with_column_type_specified(shutdown_only, tmp_path): | |||
| Version(pa.__version__) < Version("7.0.0"), | |||
| reason="invalid_row_handler was added in pyarrow 7.0.0", | |||
| ) | |||
| def test_csv_invalid_file_handler(shutdown_only, tmp_path): | |||
| def test_csv_invalid_file_handler(ray_start_regular_shared, tmp_path): | |||
There was a problem hiding this comment.
should we update other tests to use ray_start_regular_shared instead of shutdown_onl? e.g. test_csv_read_with_column_type_specified
There was a problem hiding this comment.
Yeah, we should. Can do in a follow-up PR if/when I have time
There was a problem hiding this comment.
Follow-up here: #48352
8 tasks
bveeramani
added a commit
that referenced
this pull request
Oct 29, 2024
See #48228 (comment). Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Jay-ju
pushed a commit
to Jay-ju/ray
that referenced
this pull request
Nov 5, 2024
This PR refactors several unit tests to simplify them and make them less brittle (e.g., by parameterizing inputs or removing ordering assumptions). Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Jay-ju
pushed a commit
to Jay-ju/ray
that referenced
this pull request
Nov 5, 2024
…#48352) See ray-project#48228 (comment). Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
JP-sDEV
pushed a commit
to JP-sDEV/ray
that referenced
this pull request
Nov 14, 2024
This PR refactors several unit tests to simplify them and make them less brittle (e.g., by parameterizing inputs or removing ordering assumptions). Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
JP-sDEV
pushed a commit
to JP-sDEV/ray
that referenced
this pull request
Nov 14, 2024
…#48352) See ray-project#48228 (comment). Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
mohitjain2504
pushed a commit
to mohitjain2504/ray
that referenced
this pull request
Nov 15, 2024
This PR refactors several unit tests to simplify them and make them less brittle (e.g., by parameterizing inputs or removing ordering assumptions). Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> Signed-off-by: mohitjain2504 <mohit.jain@dream11.com>
mohitjain2504
pushed a commit
to mohitjain2504/ray
that referenced
this pull request
Nov 15, 2024
…#48352) See ray-project#48228 (comment). Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> Signed-off-by: mohitjain2504 <mohit.jain@dream11.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
This PR refactors several unit tests to simplify them and make them less brittle (e.g., by parameterizing inputs or removing ordering assumptions).
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.